SPARQL’s DESCRIBE keyword is deliberately underspecified; vendors are free to do, for good or bad, whatever they want. In Stardog a DESCRIBE <theResource> query retrieves the predicates and objects for all the triples for which <theResource> is the subject. There are, of course, about seventeen thousand other ways to implement DESCRIBE; we’ve implemented four or five of them and may expose them to users in a future release of Stardog based on user feedback and requests.

Now you know and knowing is one-quarter of the fun.

Stardog supports all of the functions in SPARQL, as well as some others from XPath and SWRL. Any of these functions can be used in queries or rules. Some functions appear in multiple namespaces, but all of the namespaces will work.

See SPARQL Query Functions for the complete list.

SPARQL 1.1 Update can be used to insert RDF into or delete RDF from a Stardog database using SPARQL query forms INSERT and DELETE, respectively.

An example of deleting RDF:

Or they can be combined with WHERE clauses:

As of Stardog 2.2.4, the most efficient way to load data into Stardog is at database creation time. See the Creating a Database section for bulk loading data at database creation time. To add data to an existing Stardog database, use the add command:

The optional arguments are -f (or --format) to specify the RDF serialization type of the files to be loaded; if you specify the wrong type, add will fail. If you don’t specify a type, Stardog will try to determine the type on its own based on the file extension. For example, the files that have names ending with '.ttl' will be parsed with Turtle syntax. If you specify a type, then all the files being loaded must of that same type.

If you want to add data to a named graph, specify it via the --graph-uri or -g options.

To remove data from a Stardog database, remove is used by specifying

For example,

will remove the named graph http://foo and all its triples from myDatabase.

will remove the triples in 1.rdf from (the default graph of) myDatabase.

will remove the triples in the Turtle files 2.rdf and 3.rdf from the named graph http://foo of myDatabase.

Strict or loose parsing may be set for the input payload by using --strict-parsing=TRUE|FALSE.

RDF parsing in Stardog is strict: it requires typed RDF literals to match their explicit datatypes, URIs to be well-formed, etc. In some cases, strict parsing isn’t ideal—it may be disabled using the --strict-parsing=FALSE.

However, even with strict parsing disabled, Stardog’s RDF parser may encounter parse errors from which it cannot recover. And loading data in lax mode may lead to unexpected SPARQL query results. For example, malformed literals ("2.5"^^xsd:int) used in filter evaluation may lead to undesired results.

Commit a new version by adding and removing triples specified in files. Different from the data add/remove commands, commit allows one to add and remove triples in one commit and to associate a commit message.

To commit changes:

To see all revisions (commits) in a database:

The output can be tweaked using --after, --before, and --committer.

You can revert specific revisions, ranges, etc.

You can also see the differences between revisions; by default, between the head version and its previous versions or the changes in a specific commit, respectively:

You can also create, drop, list tags, i.e., named revisions:

The revision history of the database is represented as RDF using the W3C PROV vocabulary and can be queried using SPARQL:[7]

The indexing strategy creates a "search document" per RDF literal. Each document consists of the following fields: literal ID; literal value; and contexts.

We use the predicate http://jena.hpl.hp.com/ARQ/property#textMatch to access the search index in a SPARQL query.

For example,

This query selects the top 50 literals, and their scores, which match 'mac' and whose scores are above a threshold of 0.5. These literals are then joined with the generic BGP ?s ?p ?l to get the resources (?s) that have those literals. Alternatively, you could use ?s rdf:type ex:Book if you only wanted to select the books which reference the search criteria; you can include as many other BGPs as you like to enhance your initial search results.

First, check out the search man page:

Okay, now let’s do a search over the O’Reilly book catalog in RDF for everything mentioning "html":

The results?

Stardog search is based on Lucene 4.2.0: we support all of the search modifiers that Lucene supports, with the exception of fields.

For a more detailed discussion, see the Lucene docs.

Seriously, this is a lot more fun if you just download the damn thing and hit it with a browser!

To use the Stardog CLI tools, you can start by asking them to display help:

Or:

These work too:

We divide administrative functionality into two CLI programs for reasons of security: stardog-admin will need, in production environments, to have considerably tighter access restrictions than stardog.

The CLI tools use "command groups" to make CLI subcommands easier to find. To print help for a particular command group, just ask for help:

The command groups and their subcommands:

The main help command for either CLI tool will print a listing of the command groups:

To get more information about a particular command, simply issue the help command for it including its command group:

Finally, everything here about command groups, commands, and online help works for stardog-admin, too:

Stardog also supports CLI autocomplete via bash autocompletion. To install autocomplete for bash shell, you’ll first want to make sure bash completion is installed:

You need to know how to make a connection string to talk to a Stardog database. A connection string may consist solely of the database name in cases where

In other cases, a "fully qualified" connection string, as described below, is required.

Further, the connection string is now assumed to be the first argument of any command that requires a connection string. Some CLI subcommands require a Stardog connection string as an argument to identify the server and database upon which operations are to be performed.

Connection strings are URLs and may either be local to the machine where the CLI is run or they may be on some other remote machine.

There are two URL schemes recognized by Stardog:

The former uses Stardog’s (extended) version of SPARQL Protocol; the latter uses Stardog’s native data access protocol, called SNARL.

To make a connection string, you need to know the URL scheme; the machine name and port Stardog Server is running on; any (optional) URL path to the database;[9] and the name of the database:

Here are some example connection strings:

Using the default ports for SNARL and HTTP protocols simplifies connection strings. connectionOptions are a series of ; delimited key-value pairs which themselves are = delimited. Key names must be lowercase and their values are case-sensitive. Finally, in the case where the scheme is SNARL, the machine is "localhost", and the port is the default SNARL port, a connection string may consist of the "databaseName" only.

The process of installation is pretty simple; see the Quick Start Guide for details.

But how do we easily upgrade between versions? The key is judicious use of STARDOG_HOME. Best practice is to keep installation directories for different versions separate and use a STARDOG_HOME in another location for storing databases.[10] One you set your STARDOG_HOME environment variable to point to this directory, you can simply stop the old version and start the new version without copying or moving any files. You can also specify the home directory using the --home argument when starting the server.

To use any of these commands against a remote server, pass a global --server argument with an HTTP or SNARL URL.

See the Security section for information about Stardog’s security system, secure deployment patterns, and more.

Stardog Server’s behavior can be configured via the JVM arg stardog.home, which sets Stardog Home, overriding the value of STARDOG_HOME set as an environment variable. Stardog Server’s behavior can also be configured via a stardog.properties—which is a Java Properties file—file in STARDOG_HOME. To change the behavior of a running Stardog Server, it is necessary to restart it.

The following twiddly knobs for Stardog Server are available in stardog.properties:[11]

The simplest way to start the server—running on the default port, detaching to run as a daemon, and writing stardog.log to the current working directory— is

To specify parameters:

The port can be specified using the property --port. The HTTP interface can be disabled by using the flag --no-http and the SNARL interface via --no-snarl.

To shut the server down:

If you started Stardog on a port other than the default, or want to shut down a remote server, you can simply use the --server option to specify the location of the server to shutdown.

By default Stardog will bind it’s server to 0.0.0.0. You can specify a different network interface for Stardog to bind to using the --bind property of server start.

Stardog’s JMX implementation is called Watchdog. In addition to providing some basic JVM information, Watchdog also exports information about the Stardog DBMS configuration as well as stats for all of the databases within the system, such as the total number of open connections, size, and average query time.

Stardog Server will lock STARDOG_HOME when it starts to prevent synchronization errors and other nasties if you start more than one Stardog Server with the same STARDOG_HOME. If you need to run more than one Stardog Server instance, choose a different STARDOG_HOME or pass a different value to --home.

See the stardog.properties file (in the distribution) for a complete discussion of how access and audit logging work in Stardog Server. Basically, audit logging is a superset of the events in access logging. Access logging covers the most often required logging events; you should consider enabling audit logging if you really need to log every server event. Logging generally doesn’t have much impact on performance; but the safest way to insure that impact is negligible is to log to a separate disk (or to a centralized logging server, etc.).

The important configuration choices are whether logs should be binary or plain text (both based on ProtocolBuffer message formats); the type of logging (audit or access); the logging location (which may be "off disk" or even "off machine") Logging to a centralized logging facility requires a Java plugin that implements the Stardog Server logging interface; see Java Programming for more information; and the log rotation policy (file size or time).

Slow query logging is also available. See the Managing Queries section below.

To administer a Stardog database, some config options must be set at creation time; others may be changed subsequently and some may never be changed. All of the config options have sensible defaults (except, obviously, for the database name), so you don’t have to twiddle any of the knobs till you really need to.

The following are the legal configuration options for a Stardog database:

Databases are either online or offline; this allows database maintenance to be decoupled from server maintenance.

Stardog databases may be created locally or remotely; but, of course, performance is better if data files don’t have to be transferred over a network during creation and initial loading. See the section below about loading compressed data. All data files, indexes, and server metadata for the new database will be stored in Stardog Home. Stardog won’t create a database with the same name as an existing database. Stardog database names must conform to the regular expression, [A-Za-z]{1}[A-Za-z0-9_-].

Minimally, the only thing you must know to create a Stardog database is a database name; alternately, you may customize some other database parameters and options depending on anticipated workloads, data modeling, and other factors.

See stardog-admin help db create for all the details including examples.

Stardog database archetypes are a new feature in 2.0. A database archetype is a named, vendor-defined or user-defined bundle of data and functionality to be applied at database-creation time. Archetypes are primarily for supporting various data standards or toolchain configurations in a simple way.

For example, the SKOS standard from W3C defines an OWL vocabulary for building taxonomies, thesauruses, etc. SKOS is made up by a vocabulary, some constraints, some kinds of reasoning, and (typically) some SPARQL queries. If you are developing an app that uses SKOS, without Stardog’s SKOS archetype, you are responsible for assembling all that SKOS stuff yourself. Which is tedious, error-prone, and not very rewarding—​even when it’s done right the first time.

Rather than putting that burden on Stardog users, we’ve created database archetypes as a mechanism to collect these "bundles of stuff" which, as a developer, you can then simply attach to a particular database.

The last point to make is that archetypes are composable: you can mix-and-match them at database creation time as needed.

Stardog supports two database archetypes out-of-the-box: PROV and SKOS.

As a boon to the overworked admin or devops peeps, Stardog Server supports database creation templates: you can pass a Java Properties file with config values set and with the values (typically just the database name) that are unique to a specific database passed in CLI parameters.

If an I/O error or an index exception occurs while querying a DB, the DB might be corrupted and repaired with the repair command. If the errors occur during executing admin commands, then the system DB might have been corrupted. System database corruptions can also cause other problems including authorization errors.

This command needs exclusive access to your Stardog home directory and therefore requires the Stardog Server not to be running. This also means that the command can only be run on the machine where the Stardog home directory is located, and you will not be able to start the Stardog Server while this command is running.

If the built-in Stardog system database is corrupted, then you can use the database name system as the repair argument. To repair the database myDB:

To repair the system database:

Stardog includes both physical and logical backup utilities; logical backups are accomplished using the export CLI command. Physical backups and restores are accomplished using stardog-admin db backup and stardog-admin db restore commands, respectively.

These tools perform physical backups, including database metadata, rather than logical backups via some RDF serialization. They are native Stardog backups and can only be restored with Stardog tools. Backup may be accomplished while a database is online; backup is performed in a read transaction: reads and writes may continue, but writes performed during the backup are not reflected in the backup.

See the man pages for backup and restore for details.

SPARQL queries can be verbose; but at least the PREFIX declarations in the prologue of each query are easy to screw up! Stardog allows database administrators to persist and manage custom namespace prefix bindings:

If no files are used during database creation, or if the files do not define any prefixes (e.g. NTriples), then the "Big Four" default prefixes are stored: RDF, RDFS, XSD, and OWL.

When executing queries in the CLI, the default table format for SPARQL SELECT results will use the bindings as qnames. SPARQL CONSTRUCT query output (including export) will also use the stored prefixes. To reiterate, namespace prefix bindings are per database, not global.

By default Stardog builds extra indexes for named graphs. These additional indexes are used when SPARQL queries specify datasets using FROM and FROM NAMED. With these additional indexes, better statistics about named graphs are also computed.

Stardog may also be configured to create and to use fewer indexes, if the database is only going to be used to store RDF triples—​that is to say, if the database will not be used to store named graph information. In this mode, Stardog will maintain fewer indexes, which will result in faster database creation and faster updates without compromising query answering performance. In such databases, quads (that is: triples with named graphs or contexts specified) may still be added to these database at any time, but query performance may degrade in such cases.

To create a database which indexes only RDF triples, set the option index.named.graphs to false at database creation time. The CLI provides a shorthand option, -i or --index-triples-only, which is equivalent.

While Stardog is generally biased in favor of read performance, write performance is also important in many applications. In order to increase write performance, Stardog may be used, optionally, with a differential index.

Stardog’s differential index is used to persist additions and removals separately from the main indexes, such that updates to the database can be performed faster. Query answering takes into consideration all the data stored in the main indexes and the differential index; hence, query answers are computed as if all the data is stored in the main indexes.

There is a slight overhead for query answering with differential indexes if the differential index size gets too large. For this reason, the differential index is merged into the main indexes when its size reaches DIFF_INDEX_MAX_LIMIT. There is no benefit of differential indexes if the main index itself is small. For this reason, the differential index is not used until the main index size reaches DIFF_INDEX_MAX_LIMIT.

In most cases, the default value of the DIFF_INDEX_MAX_LIMIT parameter will work fine and doesn’t need to be changed. The corollary is that you shouldn’t change this value in a production system till you’ve tested the effects of a change in a non-production system.

Stardog supports loading data from compressed files directly: there’s no need to uncompress files before loading. Loading compressed data is the recommended way to load large input files. Stardog supports GZIP and ZIP compressions natively.[12]

This command removes a database and all associated files and metadata. This means all files on disk pertaining to the database will be deleted, so only use drop when you’re certain! Databases must be offline in order to be dropped.

It takes as its only argument a valid database name. For example,

Stardog supports integrity constraint validation as a data quality mechanism via closed world reasoning. Constraints can be specified in OWL, SWRL, and SPARQL. Please see the Validating Constraints section for more about using ICV in Stardog.

The CLI icv subcommand can be used to add, delete, or drop all constraints from an existing database. It may also be used to validate an existing database with constraints that are passed into the icv subcommand; that is, using different constraints than the ones already associated with the database.

For details of ICV usage, see stardog help icv and stardog-admin help icv. For ICV in transacted mutations of Stardog databases, see the database creation section above.

The migrate subcommand migrates an older Stardog database to the latest version of Stardog. Its only argument is the name of the database to migrate. migrate won’t necessarily work between arbitrary Stardog version, so before upgrading check the release notes for a new version carefully to see whether migration is required or possible.

will update myDatabase to the latest database format.

You can get some information about a database by running the following command:

This will return all the metadata stored about the database, including the values of configuration options used for this database instance. If you want to get the value for a specific option then you can run the following command:

Stardog includes the capability to manage running queries according to configurable policies set at run-time; this capability includes support for listing running queries; deleting running queries; reading the status of a running query; killing running queries that exceed a time threshold automatically; and logging slow queries for analysis.

Stardog is pre-configured with sensible server-wide defaults for query management parameters; these defaults may be overridden or disabled per database.

Stardog’s search service is described in Using Stardog section. However, managing search indexes is an administrative task and, thus, is described here.

There are three modes for rebuilding indexes:

This is specified when creating a database by setting the property search.reindex.mode to sync, async, or to a valid cron expression. The default is sync.

What follows is specific guidance with respect to Stardog’s transactional semantics and guarantees.[14]

Stardog tries hard to do bulk loading at database creation time in the most efficient and scalable way possible. But data loading time can vary widely, depending on factors in the data to be loaded, including the number of unique resources, etc. Here are some tuning tips that may work for you:

Stardog uses system memory aggressively and the total system memory available to Stardog is often the most important factor in performance. Stardog uses both JVM memory (heap memory) and also the operating system memory outside the JVM (off heap memory). Having more system memory available is always good; however, increasing JVM memory too close to total system memory is not usually prudent as it may tend to increase Garbage Collection (GC) time in the JVM.

The following table shows recommended JVM memory and system memory requirements for Stardog.[19]

Out of the box, Stardog CLI sets the maximum JVM memory to 2GB. This setting works fine for most small databases (up to, say, 100 million triples). As the database size increases, we recommend increasing JVM memory. You can increase the JVM memory for Stardog by setting the system property STARDOG_JAVA_ARGS using the standard JVM options. For example, you can set this property to "-Xms4g -Xmx4g -XX:MaxDirectMemorySize=8g" to increase the JVM memory to 4GB and off-heap to 8GB. We recommend setting the minimum heap size (-Xms option) as close to the max heap size (-Xmx option) as possible.

Stardog stores data on disk in a compressed format. The disk space needed for a database depends on many factors besides the number of triples, including the number of unique resources and literals in the data, average length of resource identifiers and literals, and how much the data is compressed. The following table shows typical disk space used by a Stardog database.

These numbers are given for information purposes only; the actual disk usage for a database may be significantly different in practice. Also it is important to note that the amount of disk space needed at creation time for bulk loading data is higher as temporary files will be created. The additional disk space needed at bulk loading time can be 40% to 70% of the final database size.

Disk space used by a database is non-trivially smaller if triples-only indexing is used. Triples-only indexing reduces overall disk space used by 25% on average; however, note the tradeoff: SPARQL queries involving named graphs perform significantly better with quads indexing.

The disk space used by Stardog is additive for multiple databases and there is very little disk space used other than what is required for the databases. To calculate the total disk space needed for multiple databases, one may sum the disk space needed by each database.

You can run Stardog as a Windows Service using the following configuration. Please, note, that the following assumes commands are executed from a Command Prompt with administrative privileges.

Stardog understands the use of wildcards to represent sets of resources. A wildcard is denoted with the character *. Wildcards can be used to create complex permissions; for instance, we can give a user the ability to create any database by granting it a create permission over db:*. Similarly, wildcards can be used in order to revoke multiple permissions simultaneously.

It is possible at user-creation time to specify that a given user is a superuser. Being a superuser is equivalent to having been granted an all permission over every resource, i.e., *:*. Therefore, as expected, superusers are allowed to perform any valid action over any existing (or future) resource.

When a user creates a resource, it is automatically granted delete, write, read, grant, and revoke permissions over the new resource. If the new resource is a database, then the user is additionally granted write, read, grant, and revoke permissions over icv-constraints:theDatabase and execute permission over admin:theDatabase. These latter two permissions give the owner of the database the ability to administer the ICV constraints for the database and to administer the database itself.

Do not deploy Stardog in production or in hostile environments with the default security settings.

To setup the constraints used to validate passwords when adding new users, configure the following settings in the stardog.properties configuration file.

To avoid putting passwords into scripts or environment variables, you can put them into a suitably secured password file. If no credentials are passed explicitly in CLI invocations, or you do not ask Stardog to prompt you for credentials interactively, then it will look for credentials in a password file.

On a Unix system, Stardog will look for a file called .sdpass in the home directory of the user Stardog is running as; on a Windows system, it will look for sdpass.conf in Application Data\stardog in the home directory of the user Stardog is running as. If the file is not found in these locations, Stardog will look in the location provided by the stardog.passwd.file system property.

Stardog HTTP server includes native support for SSL. The SNARL server (via snarls://) also supports SSL. To enable Stardog to optionally support SSL connections, just pass --enable-ssl to the server start command. If you want to require the server to use SSL only, that is, to reject any non-SSL connections, then use --require-ssl.

When starting from the command line, Stardog will use the standard Java properties for specifying keystore information:

These properties are checked first in stardog.properties; then in JVM args passed in from the command line, e.g. -Djavax.net.ssl.keyStorePassword=mypwd. If you’re creating a Server progammatically via ServerBuilder, you can specify values for these properties using the appropriate ServerOptions when creating the server. These values will override anything specified in stardog.properties or via normal JVM args.

Stardog HTTP client supports SSL when the https: scheme is used in the database connection string; likewise, it uses SSL for SNARL when the connection string uses the snarls: scheme. For example, the following invocation of the Stardog command line utility will initiate an SSL connection to a remote database:

If the client is unable to authenticate to the server, then the connection will fail and an error message like the following will be generated.

The most common cause of this error is that the server presented a certificate that was not issued by an authority that the client trusts. The Stardog HTTP client driver uses standard Java security components to access a store of trusted certificates. By default, it trusts a list of certificates installed with the Java runtime environment, but it can be configured to use a custom trust store.[26]

The client driver can be directed to use a specific Java KeyStore file as a trust store by setting the javax.net.ssl.trustStore system property. To address the authentication error above, that trust store should contain the issuer of the server’s certificate. Standard Java tools can create such a file. The following invocation of the keytool utility creates a new trust store named my-truststore.jks and initializes it with the certificate in my-trusted-server.crt. The tool will prompt for a passphrase to associate with the trust store. This is not used to encrypt its contents, but can be used to ensure its integrity.[27]

The following Stardog command line invocation uses the newly created truststore.

For custom Java applications that use the Stardog HTTP client driver, the system property can be set programmatically or when the JVM is initialized.

The most common deployment approach requiring a custom trust store is when a self-signed certificate is presented by the Stardog server. For connections to succeed, the Stardog client must trust the self-signed certificate. To accomplish this with the examples given above, the self-signed certificate should be in the my-trusted-server.crt file in the keytool invocation.

A client may also fail to authenticate to the server if the hostname in the Stardog database connection string does not match a name contained in the server certificate.[28]

This will cause an error message like the following

The client driver does not support connecting when there’s a mismatch; therefore, the only workarounds are to replace the server’s certificate or modify the connection string to use an alias for the same server that matches the certificate.

All of Stardog’s interfaces (API, network, and CLI) support reasoning during query evaluation.

In order to evaluate queries in Stardog using reasoning via the command line, a specific reasoning level must be specified in the connection string:

For HTTP, the reasoning level is specified with the other HTTP request parameters:

In order to use the ReasoningConnection API one needs to specify a reasoning level. See the Java Programming section for details on specifying the reasoning level programmatically.

Currently, the API has two methods:

Proof trees are a hierarchical presentation of multiple explanations (of inferences) to make data, schemas, and rules more intelligible. Proof trees[32] provide an explanation for an inference or an inconsistency as a hierarchical structure. Nodes in the proof tree may represent an assertion in a Stardog database. Multiple assertion nodes are grouped under an inferred node.

Stardog supports two different syntaxes for defining rules. The first is native Stardog Rules syntax and is based on SPARQL, so you can re-use what you already know about SPARQL to write rules. Unless you have specific requirements otherwise, you should use this syntax for user-defined rules in Stardog. The second is the de facto standard RDF/XML syntax for SWRL. It has the advantage of being supported in many tools; but it’s not fun to read or to write. You probably don’t want to use it. Better: don’t use this syntax!

Stardog Rules Syntax is basically SPARQL "basic graph patterns" (BGPs) plus some very explicit new bits (IF-THEN) to denote the head and the body of a rule.[33] You define URI prefixes in the normal way (examples below) and use regular SPARQL variables for rule variables. As you can see, some SPARQL 1.1 syntactic sugar—​property paths, especially, but also bnode syntax—​make complex Stardog Rules concise and elegant.

Stardog supports a wide variety of functions from SPARQL, XPath, SWRL, and some native Stardog functions, too. All of them may be used in either Stardog Rules syntax or in SWRL syntax. The supported functions are enumerated here.

Besides the standard RDF(S) predicates rdf:type, rdfs:subClassOf and rdfs:subPropertyOf, Stardog supports the following special built-in predicates:

Where the sp prefix binds to tag:stardog:api:property:. Stardog also recognizes sesame:directType, sesame:directSubClassOf, and sesame:strictSubClassOf predicates where the prefix sesame binds to http://www.openrdf.org/schema/sesame#.

We show what these each of these predicates means by relating them to an equivalent triple pattern; that is, you can just write the predicate rather than the (more unwieldy) triple pattern.

The predicates sp:directSubPropertyOf and sp:strictSubPropertyOf are defined analogously.

Stardog also supports a special predicate that extends the expressivity of SWRL rules. According to SWRL, you can’t create new individuals (i.e., new instances of classes) in a SWRL rule.

This restriction is well-motivated; without it, you can easily create rules that do not terminate, that is, never reach a fixed point. Stardog’s user-defined rules weakens this restriction in some crucial aspects, subject to the following restrictions, conditions, and warnings.

So despite the general restriction in SWRL, in Stardog we actually can create new individuals with a rule by using the function UUID() as follows:

This rule will create a random URI for each instance of the class :Parent and also assert that each new instance is an instance of :Person--parents are people, too!

Query rewriting has several advantages over materialization. In materialization, the data gets expanded with respect to the schema, not with respect to any actual query. And it’s the data—​all of the data—​that gets expanded, whether any actual query subsequently requires reasoning or not. The schema is used to generate new triples, typicaly when data is added or removed from the system. However, materialization introduces several thorny issues:

When a SPARQL query gets executed, each variable is bound to a URI, blank node, or to a literal to form a particular result (a collection of these results is a result set). In the context of reasoning, URIs might represent different entities: individuals, classes, properties, etc. According to the relevant standard, every variable in a SPARQL query must bind to at most one of these types of entity.

Stardog can often figure out the right entity type from the query itself (e.g., given the triple pattern ?i ?p "a literal", we know ?p is supposed to bind to a data property); however, sometimes this isn’t possible (e.g., ?s ?p ?o). In case the types can’t be determined automatically, Stardog logs a message and evaluates the query without any reasoning.

You can add one or more type triples to the query to resolve these ambiguities.[35]

These "type triples" have the form ?var a TYPE, where TYPE is a URI representing the type of entity to which the variable ?var is supposed to bind: the most common are owl:ObjectProperty or owl:DatatypeProperty; in some cases, you might want owl:NamedIndividual, or owl:Class. For instance, if you are interested in all the object properties of :i1, you can use the following query:

Since Stardog now knows that ?p should bind to an object property, we can now infer that ?o binds to an individual, so there are no ambiguities and reasoning can be performed as requested.

Stardog might be extracting the wrong schema. You have to tell Stardog where to find the schema. See database configuration options for details.

Perhaps some of the modeling constructs (a.k.a. axioms) in your database are being ignored. You can find out which axioms are being ignored by including the following line in the logging.properties file in STARDOG_HOME:

Stardog supports schema-only reasoning for OWL 2 DL, which effectively means that only TBox queries—queries that contain TBox BGPs only—will return complete query results.

As from version 2.0, SWRL rules are only taken into account using the SL reasoning level.

The OWL 2 primer is a good place to start.

A database (DB), a.k.a. ontology, is composed of two different parts: the schema or Terminological Box (TBox) and the data or Assertional Box (ABox). Analogus to relational databases, the TBox can be thought of as the schema, and the ABox as the data. In other words, the TBox is a set of axioms, whereas the ABox is a set of assertions.

As we explain in OWL 2 Profiles, the kinds of assertion and axiom that one might use for a particular database are determined by the fragment of OWL 2 to which you’d like to adhere. In general, you should choose the OWL 2 profile that most closely fits the data modeling needs of your application.

The most common data assertions are class and property assertions. Class assertions are used to state that a particular individual is an instance of a given class. Property assertions are used to state that two particular individuals (or an individual and a literal) are related via a given property. For example, suppose we have a DB MyDB₂ that contains the following data assertions. We use the usual standard prefixes for RDF(S) and OWL.

Which says that :clark_and_parsia is a company, and that :clark_and_parsia maintains :Stardog.

The most common schema axioms are subclass axioms. Subclass axioms are used to state that every instance of a particular class is also an instance of another class. For example, suppose that MyDB₂ contains the following TBox axiom:

stating that companies are a type of organization.

When reasoning is enabled, Stardog executes SPARQL queries depending on the type of Basic Graph Patterns they contain. A BGP is said to be an "ABox BGP" if it is of one of the following forms:

A BGP is said to be a TBox BGP if it is of one of the following forms:

A BGP is said to be a Hybrid BGP if it is of one of the following forms:

where term (possibly with subscripts) is either an URI or variable; uri is a URI; and ?var is a variable.

When executing a query, ABox BGPs are handled by Stardog. TBox BGPs are executed by Pellet embedded in Stardog. Hybrid BGPs by a combination of both.

Intuitively, reasoning with a DB means to make implicit knowledge explicit. There are two main use cases for reasoning: to infer implicit knowledge and to discover modeling errors.

With respect to the first use case, recall that MyDB₂ contains the following assertion and axiom:

From this DB, we can use Stardog in order to infer that :clark_and_parsia is an organization:

Using reasoning in order to infer implicit knowledge in the context of an enterprise application can lead to simpler queries. Let us suppose, for example, that MyDB₂ contains a complex class hierarchy including several types of organization (including company). Let us further suppose that our application requires to use Stardog in order to get the list of all considered organizations. If Stardog were used with reasoning, then we would need only issue the following simple query:

In contrast, if we were using Stardog with no reasoning, then we would have to issue a more complex query that considers all possible types of organization, thus coupling queries to domain knowledge in a tight way:

Which of these queries seems more loosely coupled and more resilient to change?

Stardog can also be used in order to discover modeling errors in a DB. The most common modeling errors are unsatisfiable classes and inconsistent DBs.

An unsatisfiable class is simply a class that cannot have any instances. Say, for example, that we added the following axioms to MyDB₂:

stating that companies cannot be organizations and vice versa, and that an LLC is a company and an organization. The disjointness axiom causes the class :LLC to be unsatisfiable because, for the DB to be free of any logical contradiction, there can be no instances of :LLC.

Asserting (or inferring) that an unsatisfiable class has an instance, causes the DB to be inconsistent. In the particular case of MyDB₂, we know that :clark_and_parsia is a company and an organization; therefore, we also know that it is an instance of :LLC, and as :LLC is known to be unsatisfiable, we have that MyDB₂ is inconsistent.

Using reasoning in order to discover modeling errors in the context of an enterprise application is useful in order to maintain a correct contradiction-free model of the domain. In our example, we discovered that :LLC is unsatisfiable and MyDB₂ is inconsistent, which leads us to believe that there is a modeling error in our DB. In this case, it is easy to see that the problem is the disjointness axiom between :Company and :Organization.

As explained in the OWL 2 Web Ontology Language Profiles Specification, an OWL 2 profile is a reduced version of OWL 2 that trades some expressive power for efficiency of reasoning. There are three OWL 2 profiles, each of which achieves efficiency differently.

Each profile restricts the kinds of axiom and assertion that can be used in a DB. Colloquially, QL is the least expressive of the profiles, followed by RL and EL; however, strictly speaking, no profile is more expressive than any other as they provide incomparable sets of constructs.

Stardog supports the three profiles of OWL 2. Notably, since TBox BGPs are handled completely by Pellet, Stardog supports reasoning for the whole of OWL 2 for queries containing TBox BGPs only.

This kind of constraint guarantees certain subclass and superclass (i.e., subsumption) relationships exist between instances.

These constraints control the types of domain and range instances of properties.

These constraints control whether or not an RDF instance participates in some specified relationship.

These constraints control the number of various relationships or property values.

These constraints control how instances are related to one another via properties.

Constrains may be arbitrarily complex and include many conditions.

Constraints can be created using the ConstraintFactory which provides methods for creating integrity constraints. ConstraintFactory expects your constraints, if they are defined as OWL axioms, as RDF triples (or graph). To aid in authoring constraints in OWL, ExpressionFactory is provided for building the RDF equivalent of the OWL axioms of your constraint.

You can also write your constraints in OWL in your favorite editor and load them into the database from your OWL file.

We recommend defining your constraints as OWL axioms, but you are free to define them using SPARQL SELECT queries. If you choose to define a constraint using a SPARQL SELECT query, please keep in mind that if your query returns results, those are interpreted as the violations of the integrity constraint.

An example of creating a simple constraint using ExpressionFactory:

The ICVConnection interface provides programmatic access to the ICV support in Stardog. It provides support for adding, removing and clearing integrity constraints in your database as well as methods for checking whether or not the data is valid; and when it’s not, retrieving the list of violations.

This example shows how to add an integrity constraint to a Stardog database.

Here we show how to add a set of constraints as defined in a local OWL ontology.

Checking whether or not the contents of a database are valid is easy. Once you have an ICVConnection you can simply call its isValid() method which will return whether or not the contents of the database are valid with respect to the constraints associated with that database. Similarly, you can provide some constraints to the isValid() method to see if the data in the database is invalid for those specific constraints; which can be a subset of the constraints associated with the database, or they can be new constraints you are working on.

If the data is invalid for some constraints—either the explicit constraints in your database or a new set of constraints you have authored—you can get some information about what the violation was from the SNARL IC Connection. ICVConnection.getViolationBindings() will return the constraints which are violated, and for each constraint, you can get the violations as the set of bindings that satisfied the constraint query. You can turn the bindings into the individuals which are in the violation using ICV.asIndividuals().

In addition to using the ICConnection a data oracle to tell whether or not your data is valid with respect to some constraints, you can also use Stardog’s ICV support to protect your database from invalid data by using ICV as a guard within transactions.

When guard mode for ICV is enabled in Stardog, each commit is inspected to ensure that the contents of the database are valid for the set of constraints that have been associated with it. Should someone attempt to commit data which violates one or more of the constraints defined for the database, the commit will fail and the data will not be added/removed from your database.

By default, reasoning is not used when you enable guard mode, however you are free to specify any of the reasoning types supported by Stardog when enabling guard mode. If you have provided a specific reasoning type for guard mode it will be used during validation of the integrity constraints. This means you can author your constraints with the expectation of inference results satisfying a constraint.

This illustrates how to create a persistent disk database with ICV guard mode enabled at the QL reasoning type. Guard mode can also be enabled when the database is created on the CLI.

The process of checking whether some Stardog database is valid with respect to some integrity constraints. The result of ICV is a boolean value (true if valid, false if invalid) and, optionally, an explanation of constraint violations.

A schema (or "terminology box" a.k.a., TBox) is a set of statements that define the relationships between data elements, including property and class names, their relationships, etc. In practical terms, schema statements for a Stardog database are RDF Schema and OWL 2 terms, axioms, and definitions.

All of the triples in a Stardog database that aren’t part of the schema are part of the data (or "assertional box" a.k.a. ABox).

A declarative expression of some rule or constraint which data must conform to in order to be valid. Integrity Constraints are typically domain and application specific. They can be expressed in OWL 2 (any legal syntax), SWRL rules, or (a restricted form of) SPARQL queries.

Constraints that have been associated with a Stardog database and which are used to validate the data it contains. Each Stardog may optionally have one and only one set of constraints associated with it.

Stardog ICV assumes a closed world with respect to data and constraints: that is, it assumes that all relevant data is known to it and included in a database to be validated. It interprets the meaning of Integrity Constraints in light of this assumption; if a constraint says a value must be present, the absence of that value is interpreted as a constraint violation and, hence, as invalid data.

A legal OWL 2 inference may violate or satisfy an Integrity Constraint in Stardog. In other words, you get to have your cake (OWL as a constraint language) and eat it, too (OWL as modeling or inference language). This means that constraints are applied to a Stardog database with respect to an OWL 2 profile.

OWL is a monotonic language: that means you can never add anything to a Stardog database that causes there to be fewer legal inferences. Or, put another way, the only way to decrease the number of legal inferences is to delete something.

Monotonicity interacts with ICV in the following ways:

AdminConnection provides simple programmatic access to all administrative functions available in Stardog.

As you can see, the ConnectionConfiguration in com.clarkparsia.stardog.api package class is where the initial action takes place:

The to method takes a Database Name as a string; and then connect connects to the database using all specified properties on the configuration. This class and its constructor methods are used for all of Stardog’s Java APIs: SNARL native Stardog API, Sesame, Jena, as well as HTTP and SNARL protocol. In the latter cases, you must also call server and pass it a valid URL to the Stardog server using the HTTP or SNARL protocols.

Without the call to server, ConnectionConfiguration will attempt to connect to a local, embedded version of the Stardog server. The Connection still operates in the standard client-server mode, the only difference is that the server is running in the same JVM as your application.

See the ConnectionConfiguration API docs or How to Make a Connection String for more information.

We discuss the security sytem in Stardog in Security. When logged into the Stardog DBMS you can access all security related features detailed in the security section using any of the core security interfaces for managing users, roles, and permissions.

In examples 1 and 4 above, you can see how to use SNARL in Java to interact with Stardog. The SNARL API will give the best performance overall and is the native Stardog API. It uses some Sesame domain classes but is otherwise a clean-sheet API and implementation.

The SNARL API is fluent with the aim of making code written for Stardog easier to write and easier to maintain. Most objects are easily re-used to make basic tasks with SNARL as simple as possible. We are always interested in feedback on the API, so if you have suggestions or comments, please send them to the mailing list.

Let’s take a closer look at some of the interesting parts of SNARL.

You must always enclose changes to a database within a transaction begin and commit or rollback. Changes are local until the transaction is committed or until you try and perform a query operation to inspect the state of the database within the transaction.

By default, RDF added will go into the default context unless specified otherwise. As shown, you can use Adder directly to add statements and graphs to the database; and if you want to add data from a file or input stream, you use the io, format, and stream chain of method invocations.

See the SNARL API Javadocs for all the gory details.

Let’s look at removing data via SNARL; in the example above, you can see that file or stream-based removal is symmetric to file or stream-based addition, i.e., calling remove in an io chain with a file or stream call. See the SNARL API docs for more details about finer-grained deletes, etc.

SNARL also lets us parameterize SPARQL queries. We can make a Query object by passing a SPARQL query in the constructor. Simple. Obvious.

Next, let’s set a limit for the results: aQuery.limit10; or if we want no limit, aQuery.limitQuery.NO_LIMIT. By default, there is no limit imposed on the query object; we’ll use whatever is specified in the query. But you can use limit to override any limit specified in the query, however specifying NO_LIMIT will not remove a limit specified in a query, it will only remove any limit override you’ve specified, restoring the state to the default of using whatever is in the query.

We can execute that query with executeSelect and iterate over the results. We can also rebind the "?s" variable easily: aQuery.parameter"s", aURI, which will work for all instances of "?s" in any BGP in the query, and you can specify null to remove the binding.

Query objects are re-useable, so you can create one from your original query string and alter bindings, limit, and offset in any way you see fit and re-execute the query to get the updated results.

We strongly recommend the use of SNARL’s parameterized queries over concatenating strings together in order to build your SPARQL query. This latter approach opens up the possibility for SPARQL injection attacks unless you are very careful in scrubbing your input.[40]

SNARL also supports some sugar for the classic statement-level getSPO--scars, anyone?--interactions. We ask in the first line of the snippet above for an iterator over the Stardog connection, based on aURI in the subject position. Then a while-loop, as one might expect…​You can also parameterize Getter`s by binding different positions of the `Getter which acts like a kind of RDF statement filter—​and then iterating as usual.

The snippet doesn’t show object or context parameters on a Getter, but those work, too, in the obvious way.

Stardog supports query-time reasoning using a query rewriting technique. In short, when reasoning is requested, a query is automatically rewritten to n queries, which are then executed. As we discuss below in Connection Pooling, reasoning is enabled at the Connection layer and then any queries executed over that connection are executed with reasoning enabled; you don’t need to do anything up front when you create your database if you want to use reasoning.

In this code example, you can see that it’s trivial to enable reasoning for a Connection: simply call reasoning with the appropriate constant such as ReasoningType.QL passed in. In addition to OWL2 QL, EL, and RL, Stardog supports OWL2 DL schema queries. Stardog also supports SWRL and Stardog Rules, too.

Stardog’s search system can be used from Java. The fluent Java API for searching in SNARL looks a lot like the other search interfaces: We create a Searcher instance with a fluent constructor: limit sets a limit on the results; query contains the search query, and threshold sets a minimum threshold for the results.

Then we call the search method of our Searcher instance and iterate over the results i.e., SearchResults. Last, we can use offset on an existing Searcher to grab another page of results.

Stardog also supports performing searches over the full-text index within a SPARQL query via the LARQ SPARQL syntax. This provides a powerful mechanism for querying both your RDF index and full-text index at the same time while also giving you a more performant option to the SPARQL regex filter.

SNARL Connections support obtaining a specified type of Connection. This lets you extend and enhance the features available to a Connection while maintaining the standard, simple Connection API. The Connection as method takes as a parameter the interface, which must be a sub-type of a Connection, that you would like to use. as will either return the Connection as the view you’ve specified, or it will throw an exception if the view could not be obtained for some reason.

An example of obtaining an instance of a SearchConnection to use Stardog’s full-text search support would look like this:

Please see SNARL API docs for more information.

As you can see from the code snippet, once you’ve created a ConnectionConfiguration with all the details for connecting to a Stardog database, you can wrap that in a StardogRepository which is a Stardog-specific implementation of the Sesame Repository interface. At this point, you can use the resulting Repository like any other Sesame Repository implementation. Each time you call Repository.getConnection, your original ConnectionConfiguration will be used to spawn a new connection to the database.

Stardog’s RepositoryConnection implementation will, by default, disable autoCommit status. When enabled, every single statement added or deleted via the Connection will incur the cost of a transaction, which is too heavyweight for most use cases. You can enable autoCommit and it will work as expected; but we recommend leaving it disabled.

The initialization in Jena is a bit different from either SNARL or Sesame; you can get a Jena Model instance by passing the Connection instance returned by ConnectionConfiguration to the Stardog factory, SDJenaFactory.

Jena also wants to add data to a Model one statement at a time, which can be less than ideal. To work around this restriction, we recommend adding data to a Model in a single Stardog transaction, which is initiated with aModel.begin. Then to read data into the model, we recommend using RDF/XML, since that triggers the BulkUpdateHandler in Jena or grab a BulkUpdateHandler directly from the underlying Jena graph.

The other options include using the Stardog CLI client to bulk load a Stardog database or to use SNARL for loading and then switch to Jena for other operations, processing, query, etc.

The Stardog HTTP Protocol supports SPARQL Protocol 1.1 and additional resource representations and capabilities. The Stardog HTTP API v4 is also available on Apiary: http://docs.stardog.apiary.io/. The Stardog Linked Data API (aka "Annex") is also documented on Apiary: http://docs.annex.apiary.io/.

All HTTP requests that are mutative (add or remove) must include a valid Content-Type header set to the MIME type of the request body, where "valid" is a valid MIME type for N-Triples, Trig, Trix, Turtle, NQuads, JSON-LD, or RDF/XML:

SPARQL CONSTRUCT queries must also include a Accept header set to one of these RDF serialization types.

When issuing a SELECT query the Accept header should be set to one of the valid MIME types for SELECT results:

Stardog uses the following HTTP response codes:

There are also Stardog-specific error codes in the SD-Error-Code header in the response from the server. These can be used to further clarify the reason for the failure on the server, especially in cases where it could be ambiguous. For example, if you received a 404 from the server trying to commit a transaction denoted by the path /myDb/transaction/commit/293845klf9f934…​it’s probably not clear what is missing: it’s either the transaction or the database. In this case, the value of the SD-Error-Code header will clarify.

The enumeration of SD-Error-Code values and their meanings are as follows:

In cases of error, the message body of the result will include any error information provided by the server to indicate the cause of the error.

Returns a representation of the database. As of Stardog 2.2.4, this is merely a placeholder; in a later release, this resource will serve the web console where the database can be interacted with in a browser.

Returns the number of RDF triples in the database.

The SPARQL endpoint for the database. The valid Accept types are listed above in the HTTP Headers section.

To issue SPARQL queries with reasoning over HTTP, see Using Reasoning.

The SPARQL endpoint for updating the database with SPARQL Update. The valid Accept types are application/sparql-update or application/x-www-form-urlencoded.

Returns the explanation for the execution of a query, i.e., a query plan. All the same arguments as for Query Evaluation are legal here; but the only MIME type for the Query Plan resource is text/plain.

Returns a transaction identifier resource as text/plain, which is likely to be deprecated in a future release in favor of a hypertext format. POST to begin a transaction accepts neither body nor arguments.

Returns a representation of the committed transaction; 200 means the commit was successful. Otherwise a 500 error indicates the commit failed and the text returned in the result is the failure message.

As you might expect, failed commits exit cleanly, rolling back any changes that were made to the database.

Returns a representation of the transaction after it’s been rolled back. 200 means the rollback was successful, otherwise 500 indicates the rollback failed and the text returned in the result is the failure message.

Returns a representation of a query executed within the txId transaction. Queries within transactions will be slower as extra processing is required to make the changes visible to the query. Again, the valid Accept types are listed above in the HTTP Headers section.

Returns a representation of data added to the database of the specified transaction. Accepts an optional parameter, graph-uri, which specifies the named graph the data should be added to. If a named graph is not specified, the data is added to the default (i.e., unnamed) context. The response codes are 200 for success and 500 for failure.

Returns a representation of data removed from the database within the specified transaction. Also accepts graph-uri with the analogous meaning as above--Adding Data (Transactionally). Response codes are also the same.

Removes all data from the database within the context of the transaction. 200 indicates success; 500 indicates an error. Also takes an optional parameter, graph-uri, which removes data from a named graph. To clear only the default graph, pass DEFAULT as the value of graph-uri.

Returns the explanation of the axiom which is in the body of the POST request. The request takes the axioms in any supported RDF format and returns the explanation for why that axiom was inferred as Turtle.

If the database is logically inconsistent, this returns an explanation for the inconsistency.

Returns whether or not the database is consistent w.r.t to the TBox.

Returns the integrity constraints for the specified database serialized in any supported RDF format.

Accepts a set of valid Integrity constraints serialized in any RDF format supported by Stardog and adds them to the database in an atomic action. 200 return code indicates the constraints were added successfully, 500 indicates that the constraints were not valid or unable to be added.

Accepts a set of valid Integrity constraints serialized in any RDF format supported by Stardog and removes them from the database in a single atomic action. 200 indicates the constraints were successfully remove; 500 indicates an error.

Drops all integrity constraints for a database. 200 indicates all constraints were successfully dropped; 500 indicates an error.

The body of the POST is a single integrity constraint, serialized in any supported RDF format, with Content-type set appropriately. Returns either a text/plain result containing a single SPARQL query; or it returns 400 if more than one constraint was included in the input.

Lists all the databases available.

Output JSON example:

Copies a database db to another specified db_copy.

Creates a new database; expects a multipart request with a JSON specifying database name, options and filenames followed by (optional) file contents as a multipart POST request.

Expected input (application/json):

Drops an existing database db and all the information that it contains. Goodbye Callahan!

Migrates the existing content of a legacy database to new format.

Optimize an existing database.

Request message to set an existing database database online.

Request message to set an existing database offline; receives optionally a JSON input to specify a timeout for the offline operation. When not specified, defaults to 3 minutes as the timeout; the timeout should be provided in milliseconds. The timeout is the amount of time the database will wait for existing connections to complete before going offline. This will allow open transaction to commit/rollback, open queries to complete, etc. After the timeout has expired, all remaining open connections are closed and the database goes offline.

Optional input (application/json):

Set options in the database passed through a JSON object specification, i.e. JSON Request for option values. Database options can be found here.

Expected input (application/json):

Retrieves a set of options passed via a JSON object. The JSON input has empty values for each key, but will be filled with the option values in the database in the output.

Expected input:

Output JSON example:

Adds a new user to the system; allows a configuration option for superuser as a JSON object. Superuser configuration is set as default to false. The password must be provided for the user.

Expected input:

Changes user’s password in the system. Receives input of new password as a JSON Object.

Expected input:

Verifies if user is enabled in the system.

Output JSON example:

Verifies if the user is a superuser:

Retrieves a list of users.

Output JSON example:

Retrieves the list of the roles assigned to user.

Output JSON example:

Removes a user from the system.

Enables a user in the system; expects a JSON object in the following format:

Sets roles for a given user; expects a JSON object specifying the roles for the user in the following format:

Adds the new role to the system.

Expected input:

Retrieves the list of roles registered in the system.

Output JSON example:

Retrieves users that have the role assigned.

Output JSON example:

Deletes an existing role from the system; the force parameter is a boolean flag which indicates if the delete call for the role must be forced.

Creates a new permission for a given role over a specified resource; expects input JSON Object in the following format:

Creates a new permission for a given user over a specified resource; expects input JSON Object in the following format:

Deletes a permission for a given role over a specified resource; expects input JSON Object in the following format:

Deletes a permission for a given user over a specified resource; expects input JSON Object in the following format:

Retrieves permissions assigned to the role.

Output JSON example:

Retrieves permissions assigned to the user.

Output JSON example:

Retrieves effective permissions assigned to the user.

Output JSON example:

Shuts down the Stardog Server. If successful, returns a 202 to indicate that the request was recieved and that the server will be shut down shortly.

Issue a query over the version history metadata using SPARQL. Method has the same arguments and outputs as the normal query method of a database.

Input example:

Accepts a commit message in the body of the request and performs a VCS commit of the specified transaction

Input example:

Create a tag from the given revision id with the specified commit message.

Input example:

Delete the tag with the given revision.

Input example:

Perform a revert of a revision to the specified revision with the given commit message.